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Software  Acquisition  Program  Dynamics 

The  Problem 


Poor  acquisition  program  performance  inhibits  military  performance  by 
depriving  the  warfighter  of  critical  systems  to  achieve  mission  objectives 

•  Delayed  systems  withhold  needed  capabilities 

•  Wasted  resources  drain  funding  needed  for  new  systems 

Acquisitions  fail  for  both  technical  and  non-technical  reasons;  people  issues 
drive  adverse  acquisition  dynamics 

•  Human,  organizational,  and  management  issues  drive  cost  and  schedule  overruns 

Acquisition  programs  are  complex  systems  with  structural  dynamics 

•  Feedback  in  acquisition  produces  non-linear  interactions  that  add  complexity 

•  Complex  systems  can  produce  seemingly  unpredictable  behaviors 

Misaligned  incentives  are  a  key  driver  of  poor  acquisition  outcomes 

•  Social  dilemmas  are  a  type  of  misaligned  incentives  that  has  received  much  study 

•  Social  dilemmas  occur  frequently  in  software-reliant  acquisition  programs 
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Software  Acquisition  Program  Dynamics 

Rationale 

The  Challenge 

•  Acquisition  programs  regularly  experience  recurring  cost,  schedule,  and  quality 
issues 

•  Acquisition  management  personnel  have  limited  experience  to  recognize  that 

1 .  Such  problems  are  ubiquitous 

2.  Many  programs  fail  for  the  same  reasons 

3.  There  are  known  corrective  and  preventive  techniques  to  avoid  these  problems 

The  Objective 

•  Improve  acquisition  program  staff  decision-making,  and  thus  improve  acquisition 
program  outcomes 

What  Will  Change? 

•  Gain  a  deeper  understanding  of  problematic  acquisition  dynamics  to  help  anticipate 
issues 

•  Be  able  to  develop  and  test  solution  approaches  to  evaluate  their  effectiveness 

•  Develop  better  decision-making  for  acquisition  staff,  to  produce  better  program 
outcomes 
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Software  Acquisition  Program  Dynamics 

Scenario 


1.  A  JPO  PM  has  six 
stakeholder  programs 
planning  to  use  their 
joint  infrastructure 
software... 


2.  ...but  each  program 
demands  at  least  one 
major  feature  be  added 
to  the  software  just 
for  them. 


7.  As  cost  escalates 
and  schedules  lengthen, 
participation  in  the 
joint  program  unravels 
and  collapses. 


6.  With  one  stakeholder 
gone,  the  amortized  costs 
for  the  other  programs 
increase  further — and 
another  program  leaves. 


3.  The  JPO  agrees  to  the 
additional  requirements,  for 
fear  of  losing  stakeholders 
(who  could  build  custom  software) 


5.  As  the  schedule 
slips,  one  program 
decides  to  leave  the 
joint  program  and 
develop  its  own 
custom  software. 


4.  The  additional  design 
changes  and  coding 
significantly  increase 
total  cost,  schedule, 
complexity,  and  risk. 
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After  Before 


Software  Acquisition  Program  Dynamics 

Misaligned  Incentives  and  Structural  Dynamics 


Current  software 
acquisition 
decision-makers  are 
constantly  faced  with 
choices  driven  by 
misaligned  incentives.. 


. .  .and  structural  features 
of  the  acquisition 
system  that  feed 
counter-productive 
dynamic  behaviors 
(with  costly  side-effects). 


With  little  experience  or 
training  to  deal  with 
either  type  of  challenge, 
they  make  seemingly 
reasonable  choices... 


POOR  PROGRAM 
PERFORMANCE 

...only  to  find  their 
program  slipping 
schedule,  overrunning 
cost,  and  delivering 
partial  functionality. 


r - 1 

ALIGNED 

INCENTIVES 

r - ^ 

STRUCTURAL 

DYNAMICS 

FOCUSED 

EDUCATION 

r  1 

BETTER  PROGRAM 
PERFORMANCE 

Future  software 
acquisition 
decision-makers 
can  either  align, 
or  compensate  for 
misaligned  incentives, 
and... 

o 

...avoid  or  mitigate  the 
effects  of  structural 
features,  minimizing 
their  adverse  impact  on 
programs,  given... 

o 

...focused  education  in 
techniques  based  on 
tested  solutions  and 
a  deep  understanding 
of  recurring  acquisition 
dynamics,... 

o 

...resulting  in  better 
software  acquisition 
program  cost,  schedule, 
and  quality  performance. 

i  1 
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Software  Acquisition  Program  Dynamics 

Misaligned  Incentives  in  Software  Acquisition 

Immature  Technology 

•  Government  prefers  providing  greatest  capability,  which  requires  latest  technologies 

•  Contractors  prefer  using  latest  technologies  to  boost  staff  competency  for  future  bids 

Risk  Management  Participation 

•  Management  may  not  welcome  bad  news,  viewing  it  as  the  reporter’s  fault 

•  Developers  have  an  incentive  not  to  report  risks,  placing  personal  cost  vs.  program 
benefit 

Shared  Infrastructure  Development 

•  Programs  have  an  incentive  to  wait  for  another  program  to  use  the  shared  infrastructure 
first — better  that  they  work  out  the  bugs,  than  risk  failure  of  your  program 

Joint  Programs 

•  To  meet  conflicting  requirements,  cost,  schedule,  size,  complexity,  and  risk  all  go  up 

•  Users  prefer  custom  solutions  they  control  that  are  certain  to  meet  their  needs 


Misaligned  incentives  are  ubiquitous  throughout  acquisition 
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Software  Acquisition  Program  Dynamics 

Structural  Issues  in  Software  Acquisition 

Dynamics  in  complex  acquisition  are  also  driven  by  structural  aspects  such  as 
feedback  and  delays,  which  combine  with  incentives  to  increase  complexity. 

Diverting  Staff  from  Development  to  Maintenance  (Firefighting) 

•  Rework  to  fix  defects  in  the  current  release  diverts  resources  from  the  early  design  of  the  next 
release — injecting  even  more  defects  into  it 

Driving  Staff  to  Work  Harder  (Staff  Burnout  and  Turnover) 

•  Increasing  pressure  and  long  hours  eventually  lead  to  burnout  and  turnover — which  reduce 
productivity  and  further  increase  schedule  pressure 

Testing  to  Get  Positive  Outcomes  (Happy  Path  Testing) 

•  Schedule  pressure  drives  the  need  to  make  up  lost  time,  which  can  result  in  shortcutting  quality 
processes  such  as  rigorous  testing 

Adding  Manpower  Late  Slows  Progress  (Brooks’  Law) 

•  Adding  new  people  to  a  late  software  project  to  speed  development  sounds  attractive — but  in  reality 
adds  training  overhead,  thus  causing  further  delays 

Long  Program  Duration  Grows  Schedule  (Longer  Begets  Bigger) 

•  Long  duration  allows  greater  capability  to  be  built 

•  Long  duration  drives  use  of  immature  technology  to  avoid  obsolescence 

•  Long  duration  drives  scope  creep  due  to  changing  threats  and  new  technologies 
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Software  Acquisition  Program  Dynamics 

Related  Research 


The  dynamics  of  acquisition 
apply  concepts  from  several 
disciplines  to  describe 
acquisition  problems: 

•  Social  Science 
Game  Theory 
Social  Psychology 

e  Political  Science 

#  Economics 
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Software  Acquisition  Program  Dynamics 

Solution  Approach  ^ 


General 

Qualitative  Model 


Unintended 

Consequences  R 


Problem 

Symptom 


B 


Independent  Technical 

Assessment  (ITA)  Data 

Detailed  examinations  of 
challenged  programs 
with  interviews, 
document  reviews,  and 
code  analysis 


Acquisition 
Problem  Model 


\ 


Deep 

Understanding  of 
Dynamic 
Acquisition 
Behavior 


Model-Based 
Simulation  of 
Potential 
Solutions 


Acquisition 
Qualitative  Model 


Basis  for 
Acquisition 
Instructional 
Simulations 


Firefighting:  If  design  problems  are 
found  in  the  current  release,  more 
resources  must  be  used  to  fix  them. 
This  reduces  problems,  but  now  less 
work  is  done  on  the  next  release. 
This  undermines  its  early 
development  work,  and  increases 
design  problems  in  the  next  release. 


mm 
mm 
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Software  Acquisition  Program  Dynamics 

Solution  Approach  _2 

Build  models  of  previously  identified  acquisition  archetypes  to  create  executable 
simulations  of  significant  adverse  acquisition  program  behaviors 

•  Identify  key  dynamics  that  map  to  dilemmas  in  acquisition 

•  Build  and  validate  system  dynamics  models  of  those  dynamics 

Extend  the  models  to  support  simulation  of  promising  solutions/mitigations  to 
assess  their  efficacy 

•  Identify  known  applicable  solutions  to  the  modelled  adverse  acquisition  dynamics 

•  Validate  the  ability  of  the  models  to  represent  the  acquisition  situations  and  resolutions 

Recommend  candidate  solutions/mitigations  based  on  their  performance  in  the 
simulations  at  mitigating  counter-productive  dynamics 

•  Identify  most  promising  solutions  based  on  overall  ability  to  minimize  adverse 
dynamics 

•  Use  recommended  solutions/mitigations  to  inform  educational  materials  for  program 
staff 
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Software  Acquisition  Program  Dynamics 

Social  Dilemmas  in  Acquisition 

Incentives  can  combine  to  create  social  dilemmas  where  everyone  could  be 

better  off,  but  no  one  has  incentive  to  change  their  behaviors: 

Social  Traps'  ,  i.e.,  The  Tragedy  of  the  Commons 

•  An  individual  desires  an  immediate  benefit  that  will  cost  everyone.  If  all  give  in  to  the 
same  temptation,  everyone  is  worse  off. 

•  Acguisition  Example:  A  set  of  joint  stakeholders,  each  with  mandatory  custom 
requirements,  can  make  it  impossible  for  the  program  to  deliver  on-time  and  within 
budget  unless  they  compromise  for  the  common  good. 

Social  Fences'  :  i.e.,  Producing  a  Public  Good 

•  An  individual  faces  a  near-term  cost  that  will  benefit  everyone.  Each  prefers  to  avoid 
the  cost,  but  if  all  do,  everyone  is  worse  off. 

•  Acguisition  Example:  Programs  may  choose  not  to  use  a  new  joint  subsystem 
because  of  the  risk  it  poses  to  their  program — but  if  none  do,  it  won’t  be  built. 


1  from  Cross  and  Guyer,  Social  Traps,  University  of  Michigan  Press,  1980. 
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Software  Acquisition  Program  Dynamics 

Solutions  to  the  Tragedy  of  the  Commons 

•  Authority.  Designated  authority  regulates  the  good,  restricts  overusage 
•  May  be  difficult  and  unpopular  to  enforce  a  mandate  across  organizations 
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Software  Acquisition  Program  Dynamics 

Solutions  to  the  Tragedy  of  the  Commons  _2 

•  Privatization :  Converts  shared  ownership  to  private  ownership 

•  Each  participant  has  a  strong  incentive  to  care  for  what  they  own... 

...but  privatization  defeats  the  point  of  cooperation — causing  siloed  solutions 
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•  * 


Software  Acquisition  Program  Dynamics 

Solutions  to  the  Tragedy  of  the  Commons  _3 

•  Altmistic  Punishment  Participants  can  penalize  uncooperative  partners 

•  Significantly  increases  cooperation  when  used 

•  Cost  of  using  penalty  discourages  overuse,  making  it  self-correcting 


2  from  Fehr  and  Gachter,  “Altruistic  Punishment  in  Humans,”  Nature,  2003 
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Software  Acquisition  Program  Dynamics 

Summary 

Build  on  prior  work  in  static  models  by  developing  interactive  and  executable 
models  of  acquisition  dynamics 

•  Turn  existing  software  acquisition  domain  expertise  into  a  more  usable  form 

Use  acquisition  models  to  analyze  known  adverse  software  acquisition  dynamics, 
and  test  proposed  mitigations/solutions 

•  Apply  new  and  known  solutions  to  solving  recurring  dilemmas  in  acquisition 

Use  experiential  learning  from  hands-on  simulations  to  give  DoD  acquisition  staff 
a  deeper  understanding  of  acquisition  dynamics  to  help  make  better  decisions 

•  Understand  common  side-effects  of  decisions  that  lead  to  poor  performance 

•  Let  acquisition  staff  gain  experience  through  education — not  costly  mistakes 

Build  foundation  to  test  future  mitigation/solution  approaches  to  assess  value 

•  Qualitatively  validate  new  approaches  before  applying  them  to  programs 
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Software  Acquisition  Program  Dynamics 

For  Additional  Information 


SEI  Report:  “ Success  in  Acquisition:  Using  Archetypes  to  Beat  the  Odds' ' 
SEI  Blog:  “Themes  Across  Acquisition  Programs”:  Parts  1-5 
Website:  http://www.sei.cmu.edu/acquisition/research/archetvpes.cfm 


Acquisition  Archetypes  analyze 
recurring  patterns  in  actual 
programs,  and  recommend 
interventions  and  preventive  actions 


Download  all  twelve: 

Firefighting 
The  Bow  Wave  Effect 
Everything  for  Everybody 
Underbidding  the  Contract 
Robbing  Peter  to  Pay  Paul 
PMO  vs.  Contractor  Hostility 
...and  other  titles 
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SEI  Training 

Merging  software  engineenng  research  and  real-world  problems 


We  offer  a  diverse  range  of  learning  products — including 
classroom  training,  eLearning,  certification,  and  more — to 
serve  the  needs  of  customers  and  partners  worldwide. 
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